Papers with training algorithm
A Hyperparameter Optimization Toolkit for Neural Machine Translation Research (2023.acl-demo)
Copied to clipboard
| Challenge: | Insufficient tuning of hyperparameters may lead to poor results and exaggerated results . lack of open-source support tools means that the level of rigor in hyperparametric optimization may vary widely. |
| Approach: | They propose a hyperparameter optimization toolkit for neural machine translation that is implemented as a wrapper on top of the open-source Sockeye NMT software. |
| Outcome: | The proposed toolkit is implemented as a wrapper on top of the open-source Sockeye NMT software. |
Adversarial DPO: Harnessing Harmful Data for Reducing Toxicity with Minimal Impact on Coherence and Evasiveness in Dialogue Agents (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing toxicity within large language models can negatively impact the user experience, causing performance degradation. |
| Approach: | They propose an adversarial DPO algorithm that improves direct preference optimization (DPO) by incorporating harmful data into the generative model. |
| Outcome: | The proposed training algorithm improves the model’s resilience against harmful conversations while minimizing performance degradation. |
Learning to Collaborate for Question Answering and Asking (N18-1)
Copied to clipboard
| Challenge: | Question answering (QA) and question generation (QG) are closely related tasks. |
| Approach: | They propose a training algorithm that generalizes both Generative Adversarial Network and Generating Domain-Adaptive Nets under the question answering scenario. |
| Outcome: | The proposed training algorithm generalizes both Generative Adversarial Network (GAN) and Generating Domain-Adaptive Nets (GDAN) under the question answering scenario. |
Ranking Sentences for Extractive Summarization with Reinforcement Learning (N18-1)
Copied to clipboard
| Challenge: | Abstractive summarization involves various text rewriting operations and has been identified as a sequence-to-sequence problem. |
| Approach: | They propose a novel algorithm which globally optimizes the ROUGE evaluation metric through a reinforcement learning objective. |
| Outcome: | The proposed algorithm outperforms state-of-the-art extractive and abstractive systems when evaluated automatically and by humans. |
EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets (2021.acl-long)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models require enormous computational resources and long training time. |
| Approach: | They propose an algorithm to reduce inference time and train large NLP models by slimming the self-attention and fully-connected sub-layers inside a transformer. |
| Outcome: | The proposed algorithm achieves comparable performance to standard BERT with 35 45% less training time. |
Curriculum Learning for Domain Adaptation in Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) performance drops when domains do not match and in-domain training data is scarce. |
| Approach: | They propose a curriculum learning approach to adapt generic neural machine translation models to a specific domain. |
| Outcome: | The proposed approach outperforms unadapted and adapted baselines in two domains and two language pairs. |
Analyzing Word Embedding Through Structural Equation Modeling (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing studies have shown that word embedding improves accuracy on NLP tasks. |
| Approach: | They propose a causal diagram based on the evaluation results of word embeddings using partial least squares path modeling. |
| Outcome: | The proposed model proves that word embedding contributes to solving downstream tasks. |
Noise Isn’t Always Negative: Countering Exposure Bias in Sequence-to-Sequence Inflection Models (2020.coling-main)
Copied to clipboard
| Challenge: | Morphological inflection is a sequence-to-sequence task that sees great performance when data is plentiful, but performance falls off sharply in lower-data settings. |
| Approach: | They hypothesize that teacher forcing increases the likelihood that a model too closely models its training data. |
| Outcome: | Experiments show that teacher forcing can overfit models when they enter unknown territory. |
Iterative Search for Weakly Supervised Semantic Parsing (N19-1)
Copied to clipboard
| Challenge: | Recent work has focused on training semantic parsers via weak supervision from denotations alone. |
| Approach: | They propose an iterative training algorithm that alternates between searching for consistent logical forms and maximizing the marginal likelihood of the retrieved ones. |
| Outcome: | The proposed algorithm outperforms the previous best systems on WikiTableQuestions and Cornell Natural Language Visual Reasoning (NLVR) iteratively train models that provide guidance to subsequent models to search for logical forms of increasing complexity, thus dealing with spuriousness. |
Teaching Language Models to Self-Improve through Interactive Demonstrations (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been shown to improve performance on downstream tasks by prompting them to analyze and revise their outputs. |
| Approach: | They propose a training algorithm that prompts large language models to analyze and revise their own outputs and uses this feedback to train the small model. |
| Outcome: | The proposed approach improves LLaMA-7B's performance on math and reasoning tasks by up to 7.13%. |
DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback (2025.naacl-long)
Copied to clipboard
Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd Van Steenkiste, Ranjay Krishna, Cyrus Rashtchian
| Challenge: | Text-to-Image models (T2I) still struggle to produce images that are both aesthetically pleasing and faithful to the user’s input text. |
| Approach: | They propose a training algorithm that trains T2I models to be faithful to the input text. |
| Outcome: | The proposed model improves both the semantic alignment and aesthetic appeal of two diffusion-based T2I models, evidenced by multiple benchmarks (+1.7% on TIFA, +2.9% on DSG1K, +3.4% on VILA aesthetic). |
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation suffers from exposure bias, and alternative approaches to mitigate this are under debate. |
| Approach: | They propose to reduce exposure bias by using minimum risk training to mitigate hallucinations . they find that exposure bias is more problematic under domain shift . |
| Outcome: | The proposed methods can reduce exposure bias even on in-domain test sets. |
BTS: Harmonizing Specialized Experts into a Generalist LLM (2025.emnlp-main)
Copied to clipboard
Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Nicolaus Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan, Suchin Gururangan, Mike Lewis
| Challenge: | Branch-Train-Stitch (BTS) is an efficient and flexible training algorithm for combining independently trained large language model (LLM) experts into a single, capable generalist model. |
| Approach: | They propose an efficient and flexible training algorithm for combining large language model (LLM) experts into a single, capable generalist model using lightweight stitch layers. |
| Outcome: | The proposed model can generalize to new domains despite being frozen . it yields the best generalist performance on a variety of downstream tasks, retaining the specialized capabilities of each of the experts. |
Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies have revealed a number of pathologies of neural machine translation systems. |
| Approach: | They propose to use maximum a posteriori decoding to identify the highest-scoring translation, i.e. the mode problem, to validate the model and its training algorithm. |
| Outcome: | The proposed model reproduces the statistical data well, but the beam search strays from the statistics. |
IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces (2022.emnlp-main)
Copied to clipboard
| Challenge: | a faulty cross-lingual mapping technique results in non-isomorphic word embedding spaces . we address the root-cause of this problem by increasing the relative isomorphism of word embedsing spaces. |
| Approach: | They address the root-cause of faulty cross-lingual mapping by incorporating global measures of isomorphism into the skipgram loss function. |
| Outcome: | The proposed method improves bilingual lexicon induction under domain mismatch and with training algorithm dissimilarities. |
Enhancing Variational Autoencoders with Mutual Information Neural Estimation for Text Generation (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to train variational autoencoders (VAEs) have been proposed to alleviate the posterior collapse issue in NLP tasks. |
| Approach: | They propose to introduce a mutual information term between the input and its latent variable to regularize the objective of the VAE. |
| Outcome: | The proposed model performs better on three benchmark datasets and is comparable to state-of-the-art models. |
Zero-Shot Semantic Parsing for Instructions (P19-1)
Copied to clipboard
| Challenge: | Recent years have seen an increasing number of applications that have a natural language interface, such as chatbots or "intelligent personal assistants" |
| Approach: | They propose a new training algorithm that trains a semantic parser on examples from a set of source domains and augment it with features and a logical form candidate filtering logic to support zero-shot adaptation. |
| Outcome: | The proposed framework performs better than a non-adapted parser with features and logical form candidate filtering logic. |
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on knowledge distillation have shown that not all knowledge is necessary for learning a good student model. |
| Approach: | They propose an actor-critic approach to selecting appropriate knowledge to transfer during the process of knowledge distillation. |
| Outcome: | The proposed method outperforms several strong knowledge distillation baselines significantly on the GLUE datasets. |
Optimizing Language Models with Fair and Stable Reward Composition in Reinforcement Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has developed algorithms for reinforcement learning from human feedback and AI-generated feedback. |
| Approach: | They propose a method for reinforcement learning from human feedback and AI-generated feedback that incorporates weighting, ranking, and constraining to handle disparate rewards. |
| Outcome: | The proposed method reduces disparity and enhances stability among rewards . empirical results show that the proposed method is efficient and straightforward . |
PAC-tuning: Fine-tuning Pre-trained Language Models with PAC-driven Perturbed Gradient Descent (2023.emnlp-main)
Copied to clipboard
| Challenge: | PAC-tuning is a two-stage fine-tune method for pretrained language models . PAC training minimizes the PACBayes generalization bound to learn proper parameter distribution . |
| Approach: | They propose a two-stage fine-tuning method to minimize the PAC-Bayes generalization bound . they use PAC to inject noise with variance learned in the first stage into the model parameters . |
| Outcome: | The proposed method outperforms baseline methods on 5 GLUE benchmark tasks. |
KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Task-oriented dialogs (TOD) require a model to generate a response that optimizes for task-related metrics. |
| Approach: | They propose a faster generation procedure that samples from independent next-word distributions and introduce a fine-grained reward function to help the model focus on learning key information in a dialog. |
| Outcome: | The proposed algorithm achieves state-of-the-art performance on an offline task with 15% training time reduction compared to a standard RL algorithm using auto-regressive generation. |
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Jiahang Lin, Kai Hu, Binghai Wang, Yuhao Zhou, Zhiheng Xi, Honglin Guo, Shichun Liu, Junzhe Wang, Shihan Dou, Enyu Zhou, Hang Yan, Zhenhua Han, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing work on long document visual question answering is based on Retrieval-Augmented Generation (RAG) where textual or visual content is encoded into embeddings and relevance is determined by similarity scores with respect to the original query. |
| Approach: | They propose a framework that employs an agentic, vision-aware workflow to address long document visual question answering through iterative information discovery and synthesis. |
| Outcome: | The proposed framework outperforms existing RL systems by 10.4% on the MMLongbench-Doc benchmark and demonstrates superior training performance over GRPO. |